Skip to content

Conversation

@pbruski
Copy link

@pbruski pbruski commented Jan 28, 2020

No description provided.

@pbruski pbruski force-pushed the smarter-action-shuffler branch 2 times, most recently from 1a0566f to 0b4e0f6 Compare January 28, 2020 10:44
implementation("com.atlassian.performance.tools:concurrency:[1.0.0,2.0.0)")
implementation("com.atlassian.performance:selenium-js:[1.0.0,2.0.0)")

runtimeOnly("org.jetbrains.kotlin:kotlin-reflect:${kotlinVersion}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh oh, reflection = danger zone. We're bypassing a lot of compilation-time guarantees.

}

companion object {
fun createRandomisedScenario(seededRandom: SeededRandom, actionProportions: Map<Action, Int>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the problem we're trying to solve here?

Do skipped actions malform the ActionMetric.label proportions too much? Our initial workaround was to run the test for a longer time so they get averaged out.
If that is not effective enough, then we can avoid action skips by using a self-loading Memory, e.g. pass this to an action during Memory.recall:

fun recall(): String {
    if (issueKeys.isEmpty()) {
        SearchJqlAction(jira, throwawayActionMeter, IssueJqlMemory(), this).run()
    }
}

This way you can avoid skips with local logic rather than a global order-controlling scheme.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I want to preserve the requested proportions. During our runs, ~1500 view issue calls are skipped. This looks like a minimally invasive way of achieving this, without altering the way scenarios are created.
The problem with hiding the query inside a different action is that you lose control over proportions - if there are permission issues, the JQL action can be executed as often as view issue, which we don't want. If we stick to sorting, everything is transparent and there are no surprises hiding in the implicit initialisation.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same problem affects ViewBoard, btw.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try a different question: is there any benefit to keeping a fully-randomised shuffle in the scenario?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔
I'll come back to this next week

@pbruski pbruski force-pushed the smarter-action-shuffler branch from 0b4e0f6 to c9aa5a8 Compare February 7, 2020 16:30
@pbruski pbruski force-pushed the smarter-action-shuffler branch 2 times, most recently from 4d10d91 to d9a2e73 Compare February 15, 2020 21:24
@pbruski pbruski requested a review from dagguh February 15, 2020 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants